The possibility that a malevolent artificial intelligence might pose a serious threat to humankind has become a hotly debated issue. Various high profile individuals from the physicist Stephen Hawking to the tech entrepreneur Elon Musk have warned of the danger. Which is why the field of artificial intelligence safety is emerging as an important discipline. Computer scientists have begun to analyze the unintended consequences of poorly designed AI systems, of AI systems created with faulty ethical frameworks or ones that do not share human values. But there's an important omission in this field, say independent researchers Federico Pistono and Roman Yampolskiy from the University of Louisville in Kentucky. "Nothing, to our knowledge, has been published on how to design a malevolent machine," they say.

Fiction is full of evil robots, from the Cylons of "Battlestar Galactica" to the vengeful replicants of Blade Runner to the iconic, humanity-destroying Terminators. Yet these are all robots built from good intentions, whose horrific violence is an unintended consequence of their design, rather than the explicit point. What if, instead of human folly, an artificial intelligence caused harm because a human explicitly designed it for malicious purposes? A new study, funded in part by Elon Musk, looks at the possibilities of deliberately evil machines. Titled "Unethical Research: How to Create a Malevolent Artificial Intelligence," by Roman V. Yampolskiy of the University of Louisville and futurist Federico Pistono, the short paper looks at just what harm someone could do with an actively evil program.

Artificial intelligence is quickly becoming one of the most powerful tools in the tech industry, and while AI can be used for harmless tasks like defeating world Go champions, it also has the potential for misuse. A malevolent AI would be like a computer virus on steroids, and while there are currently no known cases, researchers Federico Pistono and Roman Yampolskiy from the University of Louisville in Kentucky believe that we should already be preparing for them. Pistono and Yampolskiy have published a research paper called "Unethical Research: How to Create a Malevolent Artificial Intelligence." In it, they explain that it is entirely possible for a malevolent AI to be created in the right environment, and they lay out what sort of warning signs the cyber security industry should be looking out for. First and foremost, Pistono and Yampolskiy say that any organization interested in creating a malevolent AI would resist any form of oversight on their research.

The possibility that a malevolent artificial intelligence might pose a serious threat to humankind has become a hotly debated issue. Various high profile individuals from the physicist Stephen Hawking to the tech entrepreneur, Elon Musk, have warned of the danger. Which is why the field of artificial intelligence safety is emerging as an important discipline. Computer scientists have begun to analyse the unintended consequences of poorly designed AI systems, of AI systems created with faulty ethical frameworks or ones that do not share human values. But there's an important omission in this field, say independent researcher Federico Pistono and Roman Yampolskiy from the University of Louisville in Kentucky. "Nothing, to our knowledge, has been published on how to design a malevolent machine," they say.

The possibility that a malevolent artificial intelligence might pose a serious threat to humankind has become a hotly debated issue. Various high profile individuals from the physicist Stephen Hawking to the tech entrepreneur Elon Musk have warned of the danger. Which is why the field of artificial intelligence safety is emerging as an important discipline. Computer scientists have begun to analyze the unintended consequences of poorly designed AI systems, of AI systems created with faulty ethical frameworks or ones that do not share human values. But there's an important omission in this field, say independent researchers Federico Pistono and Roman Yampolskiy from the University of Louisville in Kentucky. "Nothing, to our knowledge, has been published on how to design a malevolent machine," they say.